Such devices and methods are used, in particular, in mechanical engineering, automotive engineering, the ceramics industry, the shoe industry, the jewelry industry, dental technology and human medicine (orthopedics) and further areas, and are, for example, employed in measurement and recording for quality control, reverse engineering, rapid prototyping, rapid milling or digital mockup.
The increasing demands for substantially complete quality control during the production process, and for the digitization of the spatial form of prototypes renders the recording of surface topographies an ever more frequently occurring measurement task. The task arises in this case of determining the coordinates of individual points of the surface of the objects to be measured in a short time.
Measurement systems known from the prior art which use image sequences and serve to determine 3D coordinates of measurement objects which can, for example, be designed as portable, hand-held and/or permanently installed systems in this case generally have a pattern projector for illuminating the measurement object with a pattern, and are therefore also sometimes referred to as pattern-projecting 3D scanners or light structures 3D scanners. The pattern projected onto the surface of the measurement object is recorded by a camera system as a further component of the measurement system.
During a measurement, the projector thus illuminates the measurement object sequentially in time with different patterns (for example parallel light and dark stripes of different widths, in particular the stripe pattern can also, for example, be rotated by 90°, for example). The camera(s) register(s) the projected stripe pattern at a known angle of view to the projection. An image is recorded with each camera for each projection pattern. A temporal sequence of different brightness values thus results for each pixel of all the cameras.
However, apart from stripes, it is also possible in this case to project other appropriate patterns such as, for example, random patterns, pseudo codes, etc. Patterns suitable for this are sufficiently known to the person skilled in the art from the state of the art. By way of example, pseudo codes enable an easier absolute assignment of object points, something which is becoming increasingly more difficult when projecting very fine stripes. Thus, for this purpose, it is possible either to project in rapid succession firstly one or more pseudo codes followed by a fine stripe pattern, or else to project in consecutive recordings, different stripe patterns which become finer in the sequence, until the desired accuracy is achieved in the resolution of measurement points on the measurement object surface.
The 3D coordinates of the measurement object surface can then be calculated from the recorded image sequence by means of image processing from the photogrammetry and/or stripe projection by using methods known to the person skilled in the art in this field. For example, such measurement methods and measurement systems are described in WO 2008/046663, DE 101 27 304 A1, DE 196 33 686 A1, or DE 10 2008 036 710 A1.
The camera system usually comprises one or more digital cameras, which are located relative to one another during a measurement in a known spatial position. In order to ensure a stable position of the cameras relative to one another, they are mostly integrated together in a common housing in a fixed fashion with known spatial positioning and alignment, in particular the cameras being aligned in such a way that the fields of view of the individual cameras for the most part overlap. Two or three cameras are often used in this case. Here, the projector can be permanently connected to the camera system (in the case of the use of separate cameras, also only to some of the cameras present in the camera system), or else be positioned completely separate from the camera system.
The desired three-dimensional coordinates of the surface are calculated in two steps in the general case, that is to say in the case when the relative positioning and alignment of the projector relative to the camera system is fixed relative to one another and therefore not already known in advance. In a first step, the coordinates of the projector are then determined as follows. The image coordinates in the camera image are known relative to a given object point. The projector corresponds to a reversed camera. The sequence of brightness values that have been measured from the image sequence for each camera pixel can be used to calculate the number of the stripe. In the simplest case, this is done via a binary code (for example a Gray code) which marks the number of the stripe as a discrete coordinate in the projector. A higher accuracy can be achieved with the so-called phase shift method, since it can determine a nondiscrete coordinate. It can be used either as a supplement of a Gray code or as an absolute-measuring heterodyne method.
Once the projector position has been determined in this way, or given that said position is already known in advance relative to the camera system, 3D coordinates of measurements points on the measurement object surface can be determined as follows—for example using the method of forward section. The stripe number in the projector corresponds to the image coordinate in the camera. The stripe number specifies a light plane in space, the image coordinate specifies a light beam. Given a known camera and projector position, it is possible to calculate the point of intersection of the plane and the straight line. This is the desired three-dimensional coordinate of the object point in the coordinate system of the sensor. The geometric position of all the image beams must be known exactly. The exact calculation of the beams is performed with the forward section known from photogrammetry.
In order to achieve higher accuracies in this measurement method for calculating the 3D coordinates, the non-ideal properties of real lens systems, which result in distortions of the image, can be adapted by a distortion correction and/or a precise calibration of the imaging properties can be performed. All the imaging properties of the projector and cameras can in this case be measured during calibration processes known to the person skilled in the art (for example a series of calibration images), from which it is possible to create a mathematical model for describing these imaging properties (for example the parameters designating the imaging properties are determined from the series of calibration images by using photogrammetric methods—in particular a bundle adjustment calculation).
In summary, it follows that in the case of the pattern projection method or the light structures 3D scanner it is necessary to illuminate the object with a sequence of light patterns in order to enable a unique determination of the depth of the measurement points in the measurement area with the aid of triangulation (forward section). Thus, in order to ensure a sufficiently high degree of accuracy with reference to the measurement result, there is mostly a need for a plurality of images (that is to say a series of images) accompanied by illumination of the measurement object with appropriate different pattern projections (that is to say with an appropriate series of patterns). In the case of hand-held systems known from the state of the art, such as, for example, the measurement device described in WO 2008/046663), the illumination sequence must take place here so quickly that a movement by the operator during the recording of the series of images does not lead to measurement errors. It must be possible for the pixels of the respective projection that are recorded by the cameras to be satisfactorily assigned to one another. Thus, the image sequence must be faster than the pattern shift or image shift caused by the operator. Since the emittable optical energy of the projector is limited by the available optical sources and by radiation protection calculations, this leads to a limitation of the detectable energy in the camera system and thus to a limitation of the measurement on the weakly reflecting measurement object surfaces. Furthermore, the projectors have a limited projection speed (frame rate). Usual maximum frame rates of such projectors are, for example, around 60 Hz.
By way of example, conventional measurement devices require a measurement period of approximately 200 ms for a measurement process involving projection of a series of patterns and recording of an image sequence of the respective patterns with the camera system (as an example: given an exposure time of 20 ms to 40 ms per image, the recording of sequences of 8 to 10 images can result in, for example, total recording times or measurement periods of between 160 ms and 400 ms per measurement position).
Various undesired effects which hinder, complicate or even frustrate the evaluation, or at least negatively influence the achievable accuracy can therefore result when the camera arrangement, the projector (and/or, if appropriate, a measuring head in which the camera arrangement and the projector are integrated) and the measurement object are not held relative to one another during a measurement process (in a measurement position) with adequate steadiness and/or with an adequately high retention of position and alignment.
Different causes may come into question for inadequate steadiness of the camera arrangement, of the projector (and/or, if appropriate, a measuring head in which the camera arrangement and the projector are integrated), or of the measurement object.
Firstly, vibrations in the measurement environment can (for example if the measurements are carried out at a production station integrated in a production line) be transmitted to the holder of the measurement object or else to a robot arm holding the measuring head, and thus lead to interfering vibrations. Consequently, there has been a need to date for complicated measures for vibration damping, or for removal to special measurement facilities, but this greatly complicates the production process (since the measurement object has to be removed from the production line and transported into the measurement facility configured appropriately therefor).
With hand-held systems, the main cause of not being held adequately steadily is, in particular, the natural tremor in the hand of the human user.
Mention may be made—on the one hand—of motion blur and/or fuzziness in individual recorded images of an image sequence as negative effects which can be caused by inadequate ability to hold the position and alignment of the camera arrangement, the projector and the measurement object relative to one another.
On the other hand, however, it can also occur that the individual images of an image sequence do not conform to one another with reference to their respective recording positions and directions relative to the measurement object (that is to say fluctuations in the recording positions and directions of the individual images in an image sequence), with the result that respective assignment of pixels in the individual images to identical measurement points on the measurement object surface is either completely frustrated or can be enabled only by an enormously high computation outlay and incorporation of information from a multiplicity of images of the same area of the measurement object surface (that is to say there can be a need for the individual images to be spatially related by subsequent calculation that is very costly, for which reason an excess of images per image sequence have in part so far been recorded to prevent this effect, their main purpose being merely a back calculation of the spatial reference of the recording position and directions of the individual images relative one to another).
In order to extend the measurement range on the measurement object (for example to measure an object in its entirety), there is often a need for a plurality of measurements one after another (from various measurement positions and various angles of view of the camera relative to the measurement object), the results of the various measurements subsequently being stitched to one another. This can be done, for example, by selecting the acquisition areas to overlap in each case for the respective measurement processes, and using the respective overlap for the appropriate combining of the 3D coordinates (that is to say point clouds) obtained for a plurality of measurement processes (that is to say, identical or similar distributions can be identified in the point clouds determined for the individual measurement processes, and the point clouds can be joined accordingly).
However, this combining process is generally extremely computationally intensive, and even with the availability of the highest processor performances, still requires a high outlay of time and energy that is not to be underestimated and is inconvenient. For example, when using a robot arm to hold and guide the measuring head it is, for example, possible thereby to achieve a reduction in the computation outlay required for the combining process by acquiring the recording positions and directions for the individual measurements with the aid of the respective robot arm position and using these for the combination as prior information (for example as boundary conditions).
Disadvantages in this case are the relatively low degree of accuracy with which the measurement position can be determined with the aid of the robot arm position and—nevertheless—the requirement for the presence of such a robot arm. Thus, the computing power necessary for combining measurement results of a plurality of measurement processes cannot be reduced in this way for hand-held measurement systems.
Further disadvantages of systems of the state of the art which use substantially coherent optical radiation for pattern illumination are local measurement inaccuracies or measurement point gaps caused by speckle fields which occur in an undesirable fashion in the respective patterns of the pattern sequence.
The European patent application having application number 10166672.5 describes in this connection a pattern projection method or a light structures 3D scanner, there being provided at the projector, at the camera system and/or at the measurement object, inertial sensors for measuring translational and rotational accelerations of the projector, of the camera system and/or of the measurement object during recording of the image sequence. These measured accelerations measured by means of the IMU are then taken into account in the computational determination of the 3D coordinates from the recorded image sequence such that movements occurring during the recording of the image sequence (which can, for example, be caused by a lack of steadiness of the measuring head in which the camera arrangement and the projector are integrated) can be compensated computationally for the determination of the 3D coordinates.